Using Bayesian Decision Networks to Play Texas Hold’em Poker

Poker is an ideal vehicle for testing automated reasoning under uncertainty. It introduces uncertainty through physical randomization by shuffling and through incomplete information about opponents’ hands. Another source of uncertainty is the limited knowledge of opponents, their betting strategies, tendencies to bluff, play conservatively, reveal weaknesses, etc. Furthermore, poker is well known as a game of psychology, with success coming from a combination of mathematical accuracy and effective prediction of one’s opponent. All of these uncertainties must be assessed accurately and combined effectively for any reasonable level of skill in the game to be achieved, since good decision making is highly sensitive to those tasks. We describe our Bayesian Poker Program (BPP), which uses a Bayesian decision network to model the program’s poker hand, the opponent’s hand and the opponent’s playing behaviour conditioned upon the hand, and betting curves to randomise betting actions. BPP has been developed incrementally for 5-card stud poker since 1993. Here we describe its adaptation to play Texas Hold’em and a variety of advances which have improved BPP’s play. We analyse the effects of hand abstraction, then present and evaluate a hybrid solution to this problem. We present improvements in bluffing and straight and flush prediction. Finally, we extend BPP’s opponent modeling.